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Simple, Effective  Computation  of  Principal  Eigenvectors  and  their  Eigenvalues 
and  Application  to  High-Resolution  Estimation  of  Frequencies 

D.W.  Tufts  and  C.D.  Melissinos 


Department  of  Electrical  Engineering 
University  of  Rhode  Island 
Kingston,  RI  02881 


Abstract 

We  present  the  results  of  an  investigation  of  the  Prony-Lanczos  (P-L) 
method  [14,38]  and  the  power  method  [39]  for  simple  computation  of 
approximations  to  a  few  eigenvectors  and  eigenvalues  of  a  Hermitian  matrix. 
We  are  motivated  by  realization  of  high-resolution  signal  processing  in  an 
integrated  circuit.  The  computational  speeds  of  the  above  methods  are 
analyzed.  They  are  completely  dependent  on  the  speed  of  a  matrix-vector 
product  operation.  If  only  a  few  eigenvalues  or  eigenvectors  are  needed,  the 
suggested  methods  can  substitute  for  the  slower  methods  of  the  LINPACK  or 
EISPACK  subroutine  libraries.  The  accuracies  of  the  suggested  methods  are 
evaluated  using  matrices  formed  from  simulated  data  consisting  of  two 
sinusoids  plus  gaussian  noise.  Comparisons  are  made  with  the  corresponding 
eigenvalues  and  eigenvectors  obtained  using  LINPACK.  Also  the  accuracies  of 
frequency  estimates  obtained  from  the  eigenvectors  are  compared. 


This  work  was  supported  in  part  by  the  Electronics  Program  of  the 


I.  Introduction 


We  are  motivated  by  the  use  of  eigenvector  decompositions  of  data 
matrices  or  estimated  covariance  matrices  for  detection  of  signals  in  noise 
and  for  estimation  of  signal  parameters.  This  has  evolved  from  early  work  of 
Liggett  [1]  and  Owsley  [2],  to  adaptive-array-detection  improvements  of 
Tufts  and  Kirsteins  [3,33]  and  high-resolution  parameter  estimators  of 
Cantoni  and  Godara  [4],  Bienve^u  and  Eopp  [5],  Owsley  [6],  Schmidt  [21]  and 
Tufts  and  Kumaresan  [7,32]. 

Principal  component  analysis,  using  principal  eigenvalues  and 
eigenvectors  of  a  matrix,  was  initiated  by  Earl  Pearson  (1901)  [8],  and 
Frisch  (1929)  [9]  in  the  problem  of  fitting  a  line,  a  plane  or  in  general  a 
subspace  to  a  scatter  of  points  in  a  higher  dimensional  space.  Eckart  and 
Young  [34]  presented  the  use  of  singular  value  decomposition  for  finding 
low-rank  approximations  to  rectangular  matrices.  C.R.  Rao  examined  the 
applications  of  principal  component  analysis  [10].  Eigenvector  analysis  is 
also  used  in  image  processing  to  provide  efficient  representations  of 
pictures  [11].  Recently,  principal  component  analysis  has  been  coupled  with 
the  Wigner  mixed  time-frequency  signal  representation  to  perform  a  variety 
of  signal  processing  operations  [28,30,31], 

Linear  Prediction  techniques  for  estimation  of  signal  parameters, which 
are  modern  variants  of  Prony's  method,  can  be  improved  using  eigenvector 
decomposition  [7],  Prony's  method  is  a  simple  procedure  for  determining  the 
values  of  parameters  of  a  linear  combination  of  exponential  functions.  Now 
"Prony's  method*  is  usually  taken  to  mean  the  least  squares  extension  of  the 
method  as  presented  by  Hildebrand  [13].  The  errors  in  signal  parameters 
which  are  estimated  by  Prony's  method  can  be  very  large  [14].  If  the  data 
is  composed  of  undamped  sinusoids,  the  forward  and  backward  prediction 


equations  and  a  prediction  order  larger  than  the  number  of  signal  components 
can  be  used  simultaneously  as  advocated  by  Nutall  [22],  Ulrych  and  Clayton 
[23],  and  Lang  and  McClellan  [24].  Tufts  and  Enmaresan  have  shown  how  one 
can  improve  such  methods  of  parameter  estimation  by  going  through  a 
preprocessing  step  before  application  of  Prony's  method  [7,15,16,17], 
The  measured  data  matrix  or  the  matrix  of  estimated  covariances  is  replaced 
by  a  matrix  of  rank  M,  which  is  the  best  least  squares  approximation  to  the 
given  matrix.  If  there  is  no  prior  information  about  the  value  of  M,  it  is 
estimated  from  the  data  using  singular  value  decompositon  (SVD). 

The  eigenvalue  problem  [37]  is  one  area  where  extensive  research  has 
been  done  and  well  established  algorithms  are  available  in  highly  optimized 
mathematical  libraries  such  as  LINPACE  and  EISPACE  [40]  .The  computational 
complexity  of  these  algorithms  is  of  order  0(N  )  where  N  is  the  size  of  the 
matrix.  They  solve  for  the  complete  set  of  eigenvalues  and  eigenvectors 
of  the  matrix  even  if  the  problem  requires  only  a  small  subset  of  them  to  be 
computed.  For  the  above  appl icat ions , only  a  few  principal  eigenvectors  and 
eigenvalues  are  needed.  Hence, we  would  like  to  use  a  method  which  uses  this 
specialization  to  reduce  the  computations. 

Tufts  and  Eumaresan  [29,32,33]  have  suggested  procedures  for 
improving  Prony's  method  without  computation  of  eigenvectors.  These  appear 
to  perform  about  the  same  as  the  more  complicated  approaches  which  use 
eigenvalue  and  eigenvector  decomposition.  The  approach  in  [29]  is  based  on 
the  results  of  Hocking  and  Leslie  for  efficient  selection  of  a  best  subset 
[25].  The  approach  of  [32]  and  [33]  is  based  on  the  simple  computations 
which  result  from  using  the  longest  possible  prediction  interval. 


Here  we  investigate  two  different  approaches  to  achieving  SVD-like 
improvement  to  Prony's  method  without  the  computational  cost  of  actually 


computing  the  SVD  or  computing  all  eigenvectors  and  eigenvalues.  The  idea 
is  to  calculate  the  few, necessary  eigenvalues  and  eigenvectors  using  the 
power  method  [39]  and  a  method  of  Lanczos  [14].  Our  derivation  of  Lanczos' 
method  stresses  the  connection  with  Prony's  method  .  The  methods  are 
analyzed  and  their  amounts  of  computation  are  calculated.  Simulations  are 
performed  and  results  are  compared  to  the  singular  value  decomposition 
method  in  LINPACK. 

II.  The  Prony-Lanczos  Method 

Let  us  assume  that  we  start  with  a  given  square ,Hermitian  matrix  A  for 
which  we  want  to  compute  the  principal  eigenvectors  and  eigenvalues.  For 
examples,  this  could  he  either  the  true  underlying,  population  covariance 
matrix  or  the  estimated  covariance  matrix  [36]  from  spatial  or  temporal 
data.  Let  us  also  define  the  eigenvectors  and  eigenvalues  associated  with 
the  matrix  A  (dimension  A=n). 

ASi  =  kju.  .  i  =  1,2, ...,n  (1) 

where  u^*’uj  =  0  ,  i  t  j 

Uj**uj  =  1  ,  i  =  j  ,  that  is  u^  are  orthonormal  vectors.  (2) 

The  asterisk  is  used  to  denote  a  complex  conjugate  transpose. 

The  characteristic  polynomial  associated  with  the  matrix  A  is  given  by 
det (A  -  XI)  =  0  (3) 

Expanding  the  determinant  we  have  the  polynomial  equation 

Xn  +  pn-1  Xn_1  +  .  .  .  +  pQ  =  0  (4) 

and  the  roots  of  this  polynomial  will  give  us  the  eigenvalues  X^  of  the 

matrix.  Ve  briefly  summarize  the  procedure  for  obtaining  the  eigenvalues  X^ 


based  on  the  Lanczos  'power  sums*  as  presented  in  [14].  We  shall  show  that 
the  eigenvalues  can  then  be  obtained  from  the  power  sums  by  Frony's  method 
[13]. 


Let  us  select  a  starting  vector  bQ.  We  assume  that  the  starting  vector 
^  has  a  non-zero  projection  on  the  eigenvectors  of  the  matrix  A 
corresponding  to  the  eigenvalues  that  we  want  to  compute. 

We  then  analyze  the  vector  bQ  in  the  reference  system  of  the  vectors 
{uf} <  which  are  the  set  of  orthonormal  eigenvectors  of  the  matrix  A: 

*o  =  T1  Bi  +  t2  52  +  •  •  •  +  Tn  5n  (5) 

where 

xi  =  (6) 

Hence,  using  equation  (1). successive  vectors  formed  by  premultiplications 

of  ^  by  powers  of  the  matrix  A  can  be  represented  as  follows  : 
bi  *  Ab0  -  Tj  ux  +  t2  \2  n2  +  .  .  .  +  rn  XQ  uQ 
b2  =  A2bQ  =  Abj  =  Xj2  Uj  +  t2  X22  uj  +  .  .  .  rn  Xn2  un  (7) 


*k+l  -  bQ  =  Abk  =  xj  Xxk  Uj  +  t2  X2k  u2  + 
Let  us  form  the  set  of  basic  scalars: 


Then  we  shall  have: 


+  T_  X  k  U_ 
n  n  — n 


(8) 


Ak 


(9) 


which  were  called  by  Lanczos  the  'weighted  power  sums*  [14]  . 

The  problem  of  obtaining  X^'s  from  the  c^'s  is  the  'problem  of  weighted 
moments'  [14],  That  is  the  problem  of  Prony  [12]  and  the  old  and  modern 
versions  of  Prony's  method  can  be  used  to  estimate  the  X^'s. 

The  prediction-error-filter  equations  of  Prony's  method  can  be  written 


as  follows: 


ir 


eg  +  c.g,  +...+C  ,g  ,  +  c  =0 
oo  11  n-1  n-1  n 

°lgo  +  C2gl  +  •  •  •  +  c„8„-i  +  c„+i  =  0 


n6n-l  n+1 


(10a) 


eg  +  c  ^,g,  + 
no  n+1  1 


*  °2o-l»n-l  *  c2n~° 


or  in  matrix  form, 

C  *  g  *  0  (10b) 

A  non-zero  solution  is  possible  if  the  determinant  of  C  is  zero. 

From  the  theory  of  Prony's  method  [13] 

g(^)  -  +  gn_i  ^in_1  +  •  •  .  +  gi  Xi  +  g0  =  0  (11) 

hence  the  polynomial  coefficient  vector  £  is  also  orthogonal  to  the  vector 
(1  li  If  ...  J1  where  X^'s  are  the  eigenvalues  of  the  matrix  A. 

Lanczos  noticed  that  Prony's  method  can  be  simplified  if  we  substitute 
the  sequence  {1  X^^  X^  .  .  .  X^n]  for  a  row  of  the  matrix  C  to  form  a  matrix 
C'.  If  we  replace  the  matrix  C  by  C’  in  (10b),  the  non-zero  vector  £  is 
still  a  solution,  because  of  (11).  Hence  the  determinant  of  C'  must  be  zero. 


det  C* 


c 


n 


=  p'(X.)  =  0 

l 


(12) 


c2n-l 


Hence,  the  X^'s  can  be  obtained  directly  by  finding  the  zeros  of  the 
polynomial  p'(z).  That  is,  Lanczos  showed  that  it  is  not  necessary  to  first 


solve  equations  (10)  for  the  prediction-error-filter  coefficients. 

Thus, in  the  absence  of  noise,  we  know  that  entering  the  weighted  power 
sums  c^  of  (8)  in  equation  (12)  and  finding  the  roots  of  the  resulting 
polynomial  will  provide  us  with  accurate  estimates  of  the  true  eigenvalues 
of  the  covariance  matrix  A.  Note  also,  that  equation  (12)  can  be 
reduced  to  a  2n<*  order  equation  involving  only  cQ,  cj,  C2«  and  still 
provide  us  with  accurate  solutions  for  our  problem  of  estimating  one  or  two 
sinusoids. 

Now,  if  our  data  is  composed  of  one  or  two  complex  sinusoids,  then  the 
(LxL)  covariance  matrix  elements  will  be  also  one  sinusoid  or  a  sum  of  two 
sinusoids,  hence  the  rank  of  the  matrix  will  be  one  or  two, respectively. 
The  eigen-decomposition  of  the  matrix  will  show  that  it  has  only  one  or  two 
non-zero  eigenvalues  and  hence  it  can  be  characterized  by  a  linear 
combination  of  one  or  two  eigenvectors,  corresponding  to  the  principal  non¬ 
zero  eigenvalues.  In  Appendix  A  it  is  shown  that  these  eigenvectors  can  be 
expressed  as  a  linear  combination  of  complex  sinusoids  which  have 
frequencies  equal  to  these  of  the  sinusoids  composing  the  data. 

Now, suppose  that  we  have  accurately  determined  a  few  e igenva lues , say 
two.Xj  and  X2»Irom  the  (nxn)  matrix  A.  We  wish  to  determine  the 
corresponding  eigenvectors.  Two  concepts  are  used  :  (a)  premultiplication  of 
a  vector  by  the  matrix  (  A-X^I  )  removes  the  it^1  eigenvector  component  of 
that  vector  and  (b)  if  a  vector  ,  to  a  good  approximation, consists  only  of  M 
eigenvector  components  , then  removing  (M-l)  of  these  components  leaves 
one, isolated  eigenvector  component. 

Let  us  consider  the  special  case  of  a  rank  two  matrix  : 

A  =  AjUjU^*  +  ^25222*  (13) 


6 


From  equations  (5)  and  (13)  we  have: 


AbQ  -  +  x2^2-2 


Then, our  preliminary.unnormalized  estimates  of  the  two  principal 
eigenvectors  are  : 


u,'  =  (A-X2l)Ab0  =  (A-X2I) 2^2-2^  = 

2  2  2 
=  TjXj  5l  +  r2X2  n2  -  t1X1X251  -  t2X2  u2  - 

=  ^1^1  (  X^“X2  )  u  1 


And  similarly  for  the  second  eigenvector  estimate  we  have 


u2  ’  -  t2^2^2-^1^52 


Normalizing  the  eigenvectors  u^1  (i=l,2)  we  can  write  (15)  and  (16)  as 


u^'  =  e^l  u^  ;  angle  of 


u2'  =  e-^2  u2  ;  ^2~  angle  of  t2^2^2-^T^ 


In  general  , given  the  required  eigenvalues  from  the  earlier  Prony 
calculation, we  estimate  an  unnormalized  eigenvector  from  the  formula 


=  T  (  A-Xjl  )  Ab0 


7 


it 


where  the  number  of  factors  m  the  product  depends  on  the  number  of 
significant  eigenvector  components  in  AbQ. 

Finally,  a  few  comments  should  be  made  on  the  selection  of  the  starting 
vector  bQ.  Our  sole  assumption  until  now  has  been  that  bQ  has  a  non-zero 
projection  on  some  eigenvector  of  A  that  we  want  to  compute.  A  good  bQ 
vector  would  have  to  be  biased  in  favor  of  the  principal  eigenvectors.  We 
have  found  that  the  Fourier  vector  provides  a  very  good  selection  for  bQ. 
This  vector  will  have  its  fundamental  frequency  computed  from  the  maximum 
peak  of  the  DFT  data  spectrum.  Very  frequently  in  signal  processing 
applications  the  data  is  preprocessed  through  a  DFT  step  for  a  coarse 
analysis.  This  is  a  valuable  bonus  for  our  method  to  use  the  available 
information  for  further  processing. 


Ill .  The  Power  Method 

Suppose  A  is  a  Eermitian  (nxn)  matrix.  The  SVD  theorem  [37]  states 
that  A  can  be  written  as: 


A  =  U  *  S  ’  UJ 


where  D  is  a  unitary  matrix  and  S  is  a  matrix  consisting  of  real  only 
diagonal  elements  [37]. 

The  power  method  computes  the  dominating  singular  vectors  one  at  a  time 


and  is  based  on  solving  the  equation: 


for  the  singular  vector  u  and  the  singular  value  s.  The  power  method  uses 
an  iterative  scheme  to  solve  (21).  We  instead  suggest  a  two-step  solution 
using  an  appropriate  starting  vector  bQ: 

Bl  -  A  b0  /  | |A  b„ | I  (22) 

The  singular  value  is  chosen  to  be: 

»1-  II*  Soil  (23) 

In  order  to  obtain  the  next  singular  vector,  the  estimated  singular  plane 
(uiui1)  is  removed  from  A  using  the  following  deflation  procedure  [37]: 

A'  =  A  -  (24) 

and  the  procedure  is  repeated  with  matrix  A  to  yield  S2«U2* 

The  selection  of  bQ  is  very  important  and  the  Fourier  vector  provides  a 
very  good  estimate.  This  preprocessing  step  can  be  implemented  in  VLSI  very 
efficiently  using  summation-by-parts  [28]  or  the  Fast  Hartley  Transform 
[42,43]  methods.  A  necessary  thing  required  to  implement  the  power  method 
is  a  circuit  capable  of  computing  matrix  vector  products  of  the  form  An. 
But  the  rounding  errors  associated  with  it  are  always  worrisome  limiting  the 
usefulness  of  the  power  method.  For  this  reason  we  propose  to  use  the 
permuted  difference  coefficients  (PDC)  algorithm  [26,27]  coupled  with  the 
known  Fourier  vector  to  perform  the  above  operation  with  high  accuracy  and 
no  round-off  errors.  A  VLSI  implementation  for  the  PDC  algorithm  can  be 
easily  realized  using  a  random  access  memory  (RAM)  toghether  with  a  read- 


only-memory  (ROM)  where  the  original  Fourier  coefficients  and  the  subsequent 
reordered  coefficients  addresses  are  stored. 


IV.  Operation  count 

In  this  section  we  calculate  the  total  operations  needed  for  the 
singular  value  decomposition  (UNPACK),  the  Prony-Lanczos  method  and  the 
Power  method. 

(1).  The  matrix  eigenvalue  problem  has  been  solved  in  both  UNPACK  and 
EISPACK  mathematical  libraries.  The  UNPACK  SVD  routine  is  presented  here. 
The  solution  can  be  divided  in  three  steps:  reduction  to  bidiagonal 
form, initialization  of  the  right  and  left  unitary  matrices  U  and  V  and  the 
iterative  reduction  to  diagonal  form. 

The  reduction  to  bidiagonal  form  has  the  following  floating  point 
multiplication  count  (for  a  square  NxN  matrix): 

2[  N3  -  N3/3] 

Approximately  the  same  number  of  additions  are  required. 

In  the  second  step  the  amount  of  work  involved  when  only  the  right-hand 
side  matrix  V  is  computed, is: 

2N3/3 

floating  point  multiplies  and  approximately  the  same  number  of  additions. 


In  the  last  step  rotations  are  used  to  reduce  the  bidiagonal  matrix  to 
diagonal  form.  Thus  the  amount  of  work  depends  on  the  total  number  of 


rotations  needed.  If  this  number  is  r,  then  we  have  the  following 


require  vector-vector  inner  products  for  a  count  of  N  multiplications  and 
(N-l)  additions  per  weight  .  Therefore  the  total  is: 

4N  flops 

The  computation  of  the  eigenvalues  from  the  (second  order)  determinant 
condition  involves  12  flops  and  one  square  root  calculation.  Finally, the 
eigenvector  computation  requires  N  flops  for  each  vector  for  a  total  of  2N 
flops. 

Hence  the  total  operation  count  for  the  Prony-Lanczos  procedure 
requires : 

(2N^+6N+12)  flops  +  1  square  root 

The  above  computations  do  not  include  the  work  required  to  select  the 
starting  vector  bD  using  a  DFT  analysis.  In  this  case, assuming  a  data 
sequence  zero  padded  to  H  points, we  shall  have: 

Mlog2M  flops 

plus  (M-l)  additions  for  the  determination  of  the  maximum  spectral  peak. 

(3).  The  power  method  computes  the  dominating  eigenvalues  and 
eigenvectors  one  pair  at  a  time  .  The  second  pair  will  be  computed  following 
a  deflation  of  A.  In  general,  the  number  of  iteration  steps  depend  on  the 
convergence  criterion  severity  .  We  instead  claim  that  two-steps  are 
generally  enough  to  provide  sufficient  accuracy.  The  Fourier  vector  is  again 
selected  as  the  starting  vector  bQ. 


<y 

The  first  eigenvalue/eigenvector  pair  requires  2N  +2N  flops.  The 

2  2 

deflation  step  requires  N  flops  and  N  floating  point  additions. 

Hence  (for  a  rank  two  matrix)  the  power  method  requires  a  total  of 

5N2+4N  flops 

plus  N*  floating  point  addtions. 

V.  Simulation  results 

Let  us  assume  that  we  have  a  data  sequence  which  is  composed  of 

uniformly  spaced  samples  of  two  closely  spaced  complex  sinusoids  in  white 
noise.  We  shall  follow  the  methods  described  earlier  in  section  II  &  III  to 
calculate  the  principal  eigenvalues  and  eigenvectors. 

The  data  sequence  is  given  by  the  equation 

x(n)  *  exp(j2nfjn  +  +  exp(j2nf2n  +  +  (25) 

with  fj  =  0.52Hz,  f2  *  0.5Hz  and  for  n=l,2,...,25 

Here,  25  data  samples  are  used  and  the  phase  difference  is  A0  =  n/2 
computed  at  the  middle  of  the  data  set,  effectively  reducing  the  signal-to- 
noise  ratio  in  that  region,  thereby  representing  the  worst  case  that  can  be 
encountered.  The  frequency  separation  is  less  than  the  reciprocal  of  the 

observation  time.  The  data  is  zero  padded  to  m-128  points  and  then  the 
maximum  peak  of  the  DFT  is  computed  to  yield  the  frequency  of  the  Fourier 

vector.  This  vector  will  be  used  as  a  starting  eigenvector  for  the  P-L  and 

Power  methods  later. 

We  construct  the  forward  plus  backward  augmented  covariance  matrix  A  of 
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size  (21x21)  .  Its  effective  rank  is  two.  The  SVD  routine  ,the‘P-L  method 
and  the  Power  method  are  employed  to  solve  for  the  eigenvalues  and 
eigenvectors  (eigenpairs)  of  the  matrix.  The  P-L  method  and  the  Power 
method  compute  only  the  two  principal  eigenpairs.  The  mean  values  and 
standard  deviations  of  the  eigenvalue  estimates  are  given  in  Table  I  for  an 
ensemble  of  500  experiments.  The  performance  of  the  P-L  and  Power  methods  is 
almost  identical  to  the  SVD  (UNPACK)  method  for  the  first  eigenvalue 
estimates.  At  high  SNR  the  second  eigenvalue  mean  and  standard  deviation 
estimate  obtained  from  the  P-L  method  is  biased  with  respect  to  the 
noiseless  SVD  results.  However  ,at  low  SNR  the  eigenvalue  statistics  are 
closer  to  the  noiseless  SVD  results  than  the  other  two  methods. 

Table  II  presents  the  statistics  of  the  distances  of  the  P-L  and  Power 
methods  eigenvectors  from  those  of  the  SVD  method.  The  distance  is  the 
inverse  cosine  of  the  angle  between  the  subspaces  spanned  by  the  estimated 
eigenvectors  [41].  The  results  show  that  for  the  first  eigenvector  the  P-L 
estimate  of  the  mean  is  less  biased  (about  one  order  of  magnitude)  than  the 
Power  method,  whereas  for  the  second  eigenvector  estimates  they  perform  the 
same.  This  shows  that  these  vectors  span  virtually  the  same  subspace  as  the 
vectors  computed  from  the  SVD  method.  The  eigenvector  estimates  were  also 
compared  to  the  signal  eigenvectors  and  the  distances  were  computed  as 
above.  The  results  show  that  at  high  SNR  the  eigenvector  spanned  subspaces 
have  a  greater  distance  from  the  signal  subspace  than  the  SVD  subspace.  At 
low  SNR  the  distance  is  reduced  and  the  second  eigenvector  statistics  are 
closer  to  the  signal  eigenvector  than  the  SVD  eigenvector. 

Table  III  shows  the  CPU  time  required  to  compute  the 
eigenvalues/eigenvectors  pairs  for  these  methods.  The  P-L  method  is  faster 
than  the  SVD  by  the  order  of  the  size  of  the  covariance  matrix,  which  here 


IS  , , 


is  21.  This  roughly  agrees  with  the  theoretical  operation  count  we  presented 
in  section  IV.  It  is  almost  twice  as  fast  as  the  Power  method.  Inclusion  of 
the  FFT  computation  in  these  two  methods  will  offset  some  of  their  speed 
advantage  over  the  SVD  .  Nevertheless  ,  the  P-L  method  is  again  about  one 
order  of  magnitude  faster  than  the  SVD  method  and  the  Power  method  a  little 
more  than  half  that  ( 6  times  faster). 

The  frequencies  f^  are  then  obtained  from  the  eigenvectors  of  the 
estimated  covariance  matrix  by  the  T-K  method  [7].  For  both  estimates  of 
the  mean  and  standard  deviation  ,  as  presented  in  Table  IV, all  three  methods 
perform  similarly  down  to  15  db.  At  Odb  the  P-L  method  yields  slightly 
better  statistics  than  the  other  two  methods. 

VI.  Cone lusion 

Two  methods, the  Prony-Lanczos  method  and  the  Power  method  are  proposed 
for  simple  computation  of  approximations  to  a  few  eigenvectors  and 
eigenvalues  of  a  Hermitian  matrix.  The  computational  speeds  of  these  methods 
were  analyzed.  The  accuracies  of  the  proposed  methods  were  evaluated  using 
covariance  matrices  from  data  consisting  of  two  sinusoids  in  a  gaussian 
noise  environment.  Comparisons  were  made  with  the  corresponding  eigenvectors 
and  eigenvalues  obtained  using  the  LINPACK  mathematical  library.  The 
suggested  methods  can  substitute  for  the  slower  method  of  LINPACK  if  a  few 
eigenvalues  or  eigenvectors  are  needed. 
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In  this  appendix  we  derive  the  eigenvalues  and  eigenvectors  of  the 
covariance  matrix  R  for  the  case  of  one  and  two  sinusoids. 

One  Complex  Sinusoid  Case: 

The  data  sequence  is  modelled  by: 

jw  n 

y(n)  =  a^  ,  n  =  1,2,..~\  ,N 

The  covariance  values  of  y(n)  are: 


N 

r^U.j)  =  — ^  ^  y  (n-i)y(n-j)  i, j=l,2, . . .  ,L 

n=L+l 


(A.l) 


Writing  the  covariance  matrix  R  explicitly  in  terms  of  the  signal,  we  have: 


l*i  I' 


Kl2*  'j"i 
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lai I  e  1  1 


,2  >1 
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2  -j«1<L-2) 


^(L-D 

lal  I  e 


hr 


(A. 2) 


We  can  diagonalize  R  by  an  orthogonal  matrix  TJ  resulting  in  the  following 


equation: 


U*RD  = 


(A. 3) 


The  eigenvalues  of  R  which  occur  along  the  diagonal  elementsnof  the  above 
equation, satisfy  the  following  equation: 


^  X.  =  tr (R)  =  L  jaj2 


(A. 4) 


But  the  covariance  matrix  R  is  of  rank=l,  since  it  has  only  one  linearly 
independent  row  (or  column).  The  rest  are  obtained  by  multiplying  by  a 


±  jo^k 

constant  number  (e  ). 


Then  the  eigenvector  corresponding  to  the  eigenvalue  Xj  =  L  |aj|^  is: 


u^  =  Cjd  e  e  e  ) 


>  \  \  '■  '• 


since  it  annihilates  every  row  of  the  matrix  (R-X.^1).  The  constant  c 
be  determined  from  the  fact  that  the  matrix  U  is  orthonormal,  hence: 


Hi*  *  Hi  =  1 


which  yields: 


1 

ci "  nr 


Hence  finally: 


1 

^1  =  H 


(1 


2jd>1 

e 


j  (L-Dfcij 

e 


(A 


and  this  is  a  Fonrier  vector  with  fundamental  frequency 
Two  Complex  Sinusoids  Case: 


The  data  sequences  is  modelled  by 


The  covariance  estimates  are  given  by  the  ezpre 
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Rewriting  the  matrix  R,  we  have: 


R  =  M1  M2 


(Lx2) (2xL) 
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If  u^  is  an  eigenvector  of  R  corresponding  to  eigenvalue  Xj,  then: 

M-j  M0  a  -j  =  (A.  9) 

Prenral tip lying  by  M2,  we  have: 

M.yM'j =  X^  M2  Uj  (A.  10) 

Thus  Xj  is  also  the  eigenvalue  of  M2M^  and  the  corresponding  eigenvector  is: 

II  =  **251  (A.  11) 

Premultiplying  (A. 10)  again  by  M^ 

MlM2MlIl  *  xlMlll  (A. 12) 

and  comparing  (A. 10)  with  (A. 12) 

uj  =  M^j  (A.  13) 

Thus  we  can  find  the  eigenvalues  and  eigenvectors  of  R  by  working  with  the 
matrix  M2M2  which  is  of  order  2.  Hence: 


.  ,2  * 

|a2|  g  +  Lx 


(A. 14) 


m2m1  = 


LjaJ2  +  xg 

JajV+Lx 


•  * 

+  x  g 


where 


L-l  . .  .  L-l 

j^2-^)n  j  Abm 

g  =  2  e  -  2  e 

n— 0  n=0 


(A. 15) 


The  eigenvalues  Xj  and  X2  ar®  found  to  bet 

Xj  =  1/2  CL  |a1 12  +  Lja2|2  +  2Re{xg}  +  ( (L  ja-j^  |2  +  L|a2|2+  (A. 16) 

2Re{xg))2-4(L2  -  |g  |2 (  |#1  |2  |a2  \2~  |x  |2 
\2  =  l/2{L|ai|2  +  L |a2  |2  +  2Re  (xg}  -  ( (L  |ax |2  +  L |a2 |2  +  2Re{xg}2)  - 
4(L2  -  |g|2)(|a1|2|a2|2  -  |x|2))1/2 

where 


Re(xg]  =  Re 


*  N 

{a2a2  }  cos(— j- 


-2)Au>  *  sin 


2 (N-L) 


s  m 


N-L 

2 _ 

~1  Ago 


A(»  sin 


LAw 


(A. 17) 


Note  that  a  column  of  the  adjoint  of  (M^-Xjl)  gives  the  eigenvector  vx  of 


MjMj. 
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Adj (M2M1-X1I) 


<L |a1  |2  -  xg)  -  k1 


(A. 18) 


-|a2|28  -  Lx* 

[~ |ax |2g-Lx  <L ia2 |2  +  x  8  )_ 

T 

Therefore  the  eigenvector  v^  is  :  v^  =  [v^  v21^ 

or  =  [ (L | a ^ |2  +  xgJ-Xj  "Jajl^g-Lxl^ 

Now  the  eigenvector  of  R  corresponding  to  X.^  is: 

H.1  =  ^1—1 
and  hence , 

Rl  =  vu  <|«l|2  *l  +  «2)  +  v21(|a2|2  e2  +  x*  e^. 
a  linear  combination  of  the  Fourier  vectors  e^  and  £2. 
Similarly,  the  eigenvector  u2  of  R  corresponding  to  X2  is: 

u2  =  v11'(ja1J2e1  +  x^)  +  v21'(|a2|2e2+x%1) 

T 

where  v^  '  =  [yn'  v2i '  ] 
and 

VH*  =  <L la2  I2  +  x*g*)-X.2 

v2l'  "  “  la2  I2  8  ~  Lx* 

The  rest  of  the  eigenvalues  of  R  are  zero  and  the  corresponding 
are  not  unique. 
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(A. 19) 


(A. 20) 


(A. 21) 


(A. 22) 

eigenvectors 
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SNR 


SVD 


P-L 


PM 


r.Lr. 
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HR 
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mean3  22.0357 

22.0126 

22.0174 

09 

st.dev=  0 

0 

0 

22.0636 

22.0423 

22.0353 

30 

0.2652 

0.2655 

0.2642 

22.0341 

22.3182 

22.2957 

15 

1.4927 

1.4936 

1.4892 

28.8561 

28.5285 

28.5425 

0 

8.6489 

8.7576 

8.7477 

Eigenvalue  estimate 


1.7107 

0.5741 

CD 

0 

0 

1.7162 

0.7504 

30 

0.0497 

0.3777 

1.8634 

1.0327 

15 

0.2856 

0.5357 

10.5379 

1.6797 

0 

2.7981 

1.2625 

Eigenvalue  estimate  X2 


TABLE 


23 


1.7131 

0 

1.7199 

0.0498 

1.8677 

0.2884 

7.3859 

2.6301 


/is 


SVD,P-L 

0 

0 

0.6980 
0.1938 
0.2775 
0.574 6 


0.5305 

0.1019 


0.4770 

- 

0 

0.5819 

- 

0.7575 

-- 

0.1991 

- 

0.1606 

- 

0.5932 

0.1035 


First  Eigenvector  Distances 


0.3917 

0 

0.1744 

0.1162 


0.4618 

0.3682 


0.8243 

0.2614 


0.6169 

0.3283 

0.2850 

0.2055 

0.1110 


0.7146 

0.2938 


Second  Eigenvector  Distances 


TABLE  II 


SNR 

SVD 

FFT 

P-L 

PM 

<30 

0.30472 

+5 

0.14050 

+4 

0.15835 

+4 

0.27610 

+4 

30 

0.24391 

+5 

0.14119 

+4 

0.15859 

+4 

0.27793 

+4 

15 

0.24538 

+5 

0.14065 

+4 

0.15877 

+4 

0.27506 

+4 

0 

0.25819 

+5 

0.14029 

+4 

0.15874 

+4 

0.27568 

+4 

Computational  Cost 

(  measured  in  time  units  ts, where  1  ts=  26.04166  (.tsec  ) 


TABLE  III 


SVD 


P-L 


PM 


mean2  0.5000 

0.5000 

0.5000 

st. dev-  0 

0 

0 

0.4999 

0.4999 

C.4999 

0.0013 

0.0013 

0.0013 

0.4961 

0.4952 

0.4962 

0.0157 

0.0137 

0.0154 

0.4331 

0.4620 

0.4551 

0.1334 

0.0898 

0.1082 

Frequency  Estimate  fj 

0.5200 

0.5200 

0.5200 

0 

0 

0 

0.5201 

0.5201 

0.5201 

0.0013 

0.0013 

0.0013 

0.5251 

0.5249 

0.5251 

0.0190 

0.0141 

0.0190 

0.5717 

0.5613 

0.5642 

0.1184 

0.0893 

0.0980 

Frequency  Estimate  f2 

TABLE  IV 
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